Listing of Claims: 



1 . (Currently Amended) A computer implemented method of detecting an imminent collision 
comprising the steps of: 

capturing and preprocessing imagery of a scene proximate a platform using an imaging 
deviee image processor ; 

producing from the imagery a depth map using a d e pth map gen e rator the image 
processor , wherein each pixel in the depth map has associated 3D position data; 

performing at a collision d e t e ctor by the image processor the steps of tessellating the 
depth map into a number of patches and selecting a plurality of the patches of the depth map for 
processing, wherein said processing comprise fitting a plane to each patch of said selected 
plurality of the patches, obtaining a normal vector to each said plane, and classifying the selected 
plurality of patches of the depth map into a plurality of classes based on the obtained normal 
vector for each said selected patch and on said 3D position data; and 

detecting a potential threat in the tessellated depth map during the processing of the 
selected plurality of the patches^ 

e stimating th e siz e of the detected potential thr e at; 

es timating th e position of th e d e t e ct e d pot e ntial thr e at; 

estimating the v o locity of th e d e t e ct e d potential thr e at; 

p e rforming a traj e ctory analy s is of the detected potential threat using th e e stimated 
position and th e e stimat e d v e locity; and 

p e rforming a collision pr e diction bas e d on th e traj e ctory analysis. 

2. (Currently Amended) The method of claim 1, further including de t e rmining-if a colli s ion is 
immin e nt bas e d on th e collision pr e diction and on the e stimat e d siz e of th e pot e ntial threat- 
comprising: 

estimating position of the detected potential threat: 
estimating velocity of the detected potential threat; 
filtering the estimated position and filtering the estimated velocity; 
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performing a trajectory analysis of the detected potential threat using the filtered 
estimated position and the filtered estimated velocity; and 

performing a collision prediction based on the trajectory analysis, 

3. (Currently Amended) The method of claim i 2, further including filt e ring the e stimat e d 
position and filt e ring th e estimat e d v e locity b e for e p e rforming traj e ctory analysis, comprising: 
estimating size of the detected potential threat; and 

determining if a collision is imminent based on the collision prediction and on the estimated size 
of the potential threat 

4. (Currently Amended) The method of claim 3- 2 wherein the filtering includes Kalman 
filtering. 

5. (Currently Amended) The method of claim i 2 wherein estimating the velocity of the 
detected potential threat includes the step of identifying 2 dim e nsional 2D feature 
correspondences from imagery produced in different time frames. 

6. (Currently Amended) The method of claim 5 wherein estimating the velocity of the detected 
potential threat further includes the step of obtaining 3D correspondences from the 2- 
dim e nsional 2D feature correspondences and from the depth map. 

7. (Original) The method of claim 6 wherein estimating the velocity of the detected potential 
threat further includes the step of estimating velocity using Random Sample Consensus. 

8. (Currently Amended) The method of claim 1 wherein said proc e ssing th e sel e ct e d plurality 
of patch e s furth e r comprises: 

fitting a plan e to e ach patch of said s e lect e d plurality of th e patch e s; 
obtaining a normal vector to each plan e , 

said plurality of classes comprise one class representing the patches that are likely to 
represent a potential threat, another class representing the patches that possibly represent a 
potential threat, and another class representing the patches that are unlikely to represent a 
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potential threat , wh e r e in said classifying is bas e d - on the obtain e d normal v e ctor for e ach patch 
and on 3D positions of each patch . 

9. (Original) The method of claim 8 further including the step of grouping patches that are 
likely to represent a potential threat together. 

10. (Original) The method of claim 9 further including the step of creating a bounding box that 
represents a potential threat, wherein the bounding box is created in accord with the grouping of 
patches. 

11. (Previously Presented) The method of claim 8 wherein detecting a potential threat in the 
tessellated depth map includes the steps of moving each patch after local tessellation to find the 
region of maximum depth map data density near the original patch location, of discarding a patch 
if the region of maximum depth map data density does not meet a predetermined criterion, and 
adding the patch to the plurality of the patches if the patch density meets the predetermined 
criterion. 

12. (Currently Amended) The method of claim & I wherein obtaining a normal vector includes 
the steps of calculating the third Eigen-vector of a matrix of patch values using a singular valued 
decomposition of the matrix, and then estimating the normal vector as the third Eigen-vector. 

13. (Currently Amended) A collision detection system, comprising: 

an imaging device for providing imagery of a scene proximate a platform; 
an image preprocessor for preprocessing said imagery; 

a depth map generator for producing a depth map from said preprocessed images 
wherein each pixel in the depth map has associated 3D position data; and 

a collision detector for tessellating the depth map into a number of patches, selecting a 
plurality of the patches of the depth map for processing, wherein said processing comprise fitting 
a plane to each patch of said selected plurality of the patches, obtaining a normal vector to each 
said plane, and classifying the selected plurality of patches of the depth map into a plurality of 
classes based on the obtained normal vector for each said selected patch and on said 3D position 
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data; and detecting a potential threat in said tessellated depth map during the processing of the 
selected plurality of the patches^ . 

wh e r e in said - collision detector e stimates s ize, position, and v e locity of said d e t e ct ed 
pot e ntial thr e at; 

wh e r e in said collision d e t e ctor p e rforms a traj e ctory analysis of said det e cted pot e ntial 

threat using s aid ostimatcd - po s ition - and -s aid -es timat o d v o locity; 

wh e r e in said collision d e t e ctor pr e dicts a colli s ion bas e d on said traj e ctory analysis; and 
wh e r e in said collision d e t e ctor d e t e rmines if a collision is immin e nt based on said 

collision pr e diction and on said estimat e d siz e . 

14. (Currently Amended) The system of claim 13, wherein said collision detector estimates 
position and velocity of said detected potential threat; said collision detector includes a filter for 
filtering image noise and outliers from said estimated position and from said estimated velocity; 
b e fore p e rforming trajectory analy s i s said collision detector performs a trajectory analysis of said 
detected potential threat using said estimated position and said estimated velocity; said collision 
detector predicts a collision based on said trajectory analysis. 

15. (Currently Amended) The system of claim 43 14 wherein said collision detector estimates 
said velocity by identifying 2 - dimensional 2D feature correspondences from imagery produced 
in different time frames and th e n obtains 3D corre s pondences of said 2 - dimen s ional feature 
correspond e nc e s using said d e pth map . 

16. (Previously Presented) The system of claim 13, further including a host vehicle, wherein 
said image preprocessor comprises a stereo image preprocessor and said imaging device 
comprises a stereo camera pair mounted in fixed locations relative to said host vehicle . 

17. (Currently Amended) The system of claim 13 wher e in said collision det e ctor d e t e cts th e 
potential-throat in th e t e ss e llat e d d e pth map by: 

fitting a piano to th e s e l e ct e d plurality of said patch e s; 
obtaining normal v e ctors to said select e d plurality of patch e s, 
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comprise as likely to represent a potential threat, as possibly representing a potential threat, or as 
being unlikely to represent a potential threat; and 

said collision detector detects the potential threat by grouping patches that are likely to 
represent a potential threat together; and forming a bounding box around said potential threat 
based on said patch groupings. 

18. (Previously Presented) The system of claim 17 wherein said collision detector searches each 
patch after tessellation to find a densest part of said patch, discards said patch if said patch 
density does not meet a predetermined criterion, and adds said patch to said plurality of patches 
if said patch density meets said predetermined criterion. 

19. (Currently Amended) The system of claim iT- 13 wherein obtaining normal vectors includes 
the steps of calculating for each patch a third Eigen-vector of a matrix of patch values using a 
singular valued decomposition of said matrix, and then estimating said normal vector as said 
third Eigen-vector. 

20. (Currently Amended) A computer readable medium having stored thereon a plurality of 
instructions, the plurality of instruction including instructions which, when executed by a 
processor causes the processor to perform the steps comprising: 

capturing and preprocessing an imagery of a scene proximate a platform; 

producing from the imagery a depth map, wherein each pixel in the depth map has 
associated 3D position data; 

tessellating the depth map into a number of patches and selecting a plurality of the 
patches of the depth map for processing, wherein said processing comprise fitting a plane to each 
patch of said selected plurality of patches, obtaining a normal vector to each said plane, and 
classifying the selected plurality of patches of the depth map into a plurality of classes based on 
the obtained normal vector for each said selected patch and on said 3D position data; 

detecting a potential threat in the tessellated depth map during the processing of the 
selected plurality of the patches % . 



estimating th e size of th e d e t e ct e d pot e ntial threat; 
estimating the position of the - detootod potential threat; 
e stimating th e v e locity of th e d e t e ct e d pot e ntial thr e at; 

p e rforming a trajectory analysis of the d e t e ct e d potential thr e at using th e e stimated 
position and the estimated velocity; and 

performing a collision pr e diction has e d on th e traj e ctory analysis. 

21. (Currently Amended) The computer readable medium of claim 20 that further causes the 
processor to estimate the size of the detected potential threat; estimate the position of the 
detected potential threat; estimate the velocity of the detected potential threat; filter the estimated 
position and the estimated velocity b e for e p e rforming traj e ctory analysis, perform a trajectory 
analysis of the detected potential threat using the estimated position and the estimated velocity; 
and perform a collision prediction based on the trajectory analysis. 

22. (Currently Amended) The computer readable medium of claim 20 2J_ that further causes the 
processor to d e t e rmin e estimate the velocity by identifying 2 - dimensional 2D feature 
correspondences from imagery produced in different time frames. 

23. (Currently Amended) The computer readable medium of claim 22 that further causes the 
processor to d e t e rmin e estimate velocity by obtaining 3D correspondences from the 2- 
dim e nsional 2D feature correspondences and from the depth map. 

24. (Currently Amended) The computer readable medium of claim 20 that furth e r caus e s th e 
processor to d e t e ct th e pot e ntial thr e at by th e step s of -: 

fitting a piano to e ach patch of th e s e l e ct e d plurality of patche s ; 
obtaining a normal vector to e ach plane, 

wherein said classifying is based on said normal v e ctor for that patch, plurality of classes 
comprise as likely to represent a potential threat, as possibly representing a potential threat, or as 
being unlikely to represent a potential threat. 
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25. (Original) The computer readable medium of claim 24 that further controls a computer to 
group patches together that are likely to represent said potential threat. 

26. (Original) The computer readable medium of claim 25 that further causes the processor to 
create a bounding box that represents the potential threat, wherein the bounding box is created in 
accord with the patch groupings. 

27. (Previously Presented) The computer readable medium of claim 20 that further causes the 
processor to detect a potential threat in said tessellated depth map by the steps of moving each 
patch after local tessellation to find the region of maximum depth map data density near the 
original patch location, discarding said patch if the region of maximum depth map data density 
does not meet a predetermined criterion, and adding said patch to said plurality of patches if said 
patch density meets said predetermined criterion. 

28. (Original) The computer readable medium of claim 20 that further causes the processor to 
obtain normal vectors by the steps of calculating for each patch a third Eigen- vector of a matrix 
of patch values using a singular valued decomposition of said matrix, and then estimating said 
normal vector as said third Eigen- vector. 

29. (New) The system of claim 15 wherein the collision detector further estimates said 
velocity by obtaining 3D correspondences from the 2D feature correspondences and from the 
depth map. 

30. (New) The system of claim 14 wherein said collision detector estimates size of said 
detected potential threat and determines if a collision is imminent based on said collision 
prediction and on said estimated size. 
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